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I. Introduction 



As presently planned, the evaluation of student charac- 
teristics related to media, presentation design, and academic 
performance is unlikely to bring much return in terms of 
replicable or generalizable results. There are a number of 
factors leading to such a conclusion, as will be detailed 
below. It is the purpose of this technical paper to present 
an alternative plan for the study of student characteristics 
designed to remedy many of the difficulties involved, and in 
addition to provide the Naval Academy with valuable by- 
products in the form of generally useful information about 
the nature of the midshipman'population. 
A. Difficulties of Regression Analysis 

The Naval Academy student body is, without question, a 
highly selected and unusual group in relation to the general 
population of college males. Scores of the midshipman population, 
undoubtedly come from relatively restricted portions of the 
scale on many tests of aptitude, achievement, interest, and 
personality. As is well known (Lord and Novick, 1968), such 
restricted samples will tend to show little or ncJ^relationship 
between variables which may be highly correlated in^ the general 
population. Thu^ it will be likely that few useful relation- 
ships will be demonstrated involving those variables for 
which the distribution of midshipman scores differ substantially 
from the general population. Nor is it likely that relationships 
demonstrated between the remaining variables are c'.iaracteristic 
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of the generaL4iQpylation. Although there are methods 
availab-le designed to correct statistically for some of 
these effects, all are based upon precarious assumptions 
and are difficult to apply and interpret correctly. The 
difficulties are further compounded by the large number of 
indep -ndent variables which are to be investigated, and 
which should be taken into account in any attempt at 
statistical correqtion. Statistical corrections c^re there- 
fore not a satisfactory general solution to the problems 
• produced by biased sampling. To summarize, the corxlusions 
regarding correlational findings must be closely limited 
to the midshipman population. 

Even disregarding the question of generality, the large 
number of independent variables to be investigated and the 
smaill samples to be used in measuring the dependent variables 
raise serious doubts as to the reliability of findings even 
in relation to the midshipman population. With a large number of 
variables, a sizable number of significant relationships can 
be expected to occur by chance at the standard level of signifi- 
cance. Furthermore, many of the subject variables will prove to 
be multiple-regression techniques to discover those "variables 
which contribute to a prediction battery but show little or no 
first order correlation with the criterion measures. With the 
additional variability associated with the use of a small sample, 
many variables which should show strong first-order correlations 
and/or contributions to a prediction battery will remain 
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uiidiscovered through lack of power. Thus, the likelihood of 
cross-validating these results in further experimentation is 
relatively low. 

Ordinarily, it is recommended that correlational studies 
be conducted with a sample size which exceeds considerably 
•the number of variables to be investigated. One means of 
limiting the number of variables commensurate with the saruple 
sizes contemplated might be to restrict attention to variables 
where strong relationship can be expected on an a priori basis. 
However, this strategy would result in a serious loss of 
information about valuable but unanticipated relationships. 
A second possible approach is to reduce the number of vari- 
ables through factor analysis. This would be feasible since 
independent variable data Vill be available on the entire 
midshipman population, but in view of the variety of factorial 
domains included in the battery the number of factors 
remaining after analysis may still be expected to be large. 
Both procedures are invalidated, as well, by the fact that 
the experimental subjects will not be a random or representa- 
tive sample of the midshipman population, so that results based 
on this sample cannot be expected to hold in that population. 
B. Advantages of Typological Approach 

The alternative plan of attack to be presented below is 
designed to capitalize on the sources of difficulty inherent 
in the situation as described above, and to turn^ these disadvantages 
into advantages. If carried out as described, the results of 
this plan will not only permit a relatively direct, reliable 



and meaningful evaluation of the relationships of subject 
characteristics as a built-in feature of future experimental 
designs, but also provide the Academy with a detailed and 
organized description of the midshipman population which can prove 
to be of exceptional general value for administrative, 
predictive, counseling, and further experimental purposes. 

When samples are small and potentially biased, the usual 
method of studying individual difference variables and their 
interactions with experimental variables is to divide the 
subjects into groups or strata which are relatively homo- 
geneous in terms of the individual difference variable and 
to employ these groups' as levels of an orthogonal factor in 
the experimental design. This procedure serves to increase 
the precision of the experiment by removing variance from, 
error into the main effect and interactions of the grouping 
variable, and provides results which are characteristic, at 
least, of those strata of the. population represented in the 
experiment. A similar approach is suggested here, but because 
*of the many variables involved, the stratification must be based 
upon an extensive preliminary analysis of the midshipman population 
designed to identify clusters or types of persons based on the 
simultaneous evaluation of the entire battery of test scores. 

The fundamental premise of this approach is that the highly 
selected nature of the midshipman population insures that relatively 
few types of individuals are represented in that population. 
In this context, a type is defined to be a subgroup of a popu- 
lation, the individuals of this subgroup all having highly 



similar profiles, or patterns>of scores on the entire 
battery of variables, but whose profiles or patterns differ 
considerably from those of other subgroups. Assuming that 
such types can be identified and described, the solution to 
the immediate experimental problem is very straightforward. 
Most of the major types of the midshipman population can be 
represented stratigraphically as levels in the experimental 
designs and differences in performance between strata can be 
determined, as well as interactions with other variables. 
The results are then directly referable to comparable types 
in the general as well as the midshipman population. This 
general method of evaluating the contribution of individual 
differences does not preclude additional correlational 
analysis where strong prior hypotheses exist, but does insure 
that information on unanticipated differences is obtained. 

In addition, the information gained on the nature and 
frequency of types present in the midshipman population would 
clearly be of extraordinary value to the Academy in numerous 
applications. Such information would provide the basrs for 
stratification in other research conducted at the Academy, 
and may be found useful in grouping classes, diagnosing 
patterns of success or failure, predicting success in future 
naval occupatronal specialties, identifying valuable types 
which are underrepresented in the Academy, and in nany other 
ways which cannot be anticipated at present. In short, a $ 
midshipman typology may be a generally useful tool of personnel 

management, well worth th: cost for these purposes even if 
not for the Leadership project alone. Ultimately, the 



typological analysis might prove more valuable to th'e Academy 
than the Leadership Course itself . 
C. Recommended Administrative Steps 

Many methods for discovering classification systems and 
defining types have been proposed. None are definitive 
mathematically, and all require considerable art in application 
and interpretation, and are entirely beyond the competence of 
the average psychological investigator. Thus it is absolutely 
essential that the, work on the typology be subcontracted to 
one or more prominent individuals, highly sophisticated and 
experienced in multivariate psychology, and particularly with 
previous experience in classif icatory analysis. It is 
recommended that the following steps should be undertaken: 

1. At lea<st six months prior to the initiation of 
. experimentation in which the results of the 

classification analysis are to be used, the qualified 
individuals (or others whom they might suggest) should 
be invited to submit a plan and bid for the analysis. 
Those to be approached should include R. B. Cattel, 
W. A. Gibson, L. L. McQuitty, J. H. Ward, A. C. 
Johnson and R. A. Shepard. 

2. One or, preferably, two bids should be accepted, 

and data provided for analysis of the entire midshiprtan 
population for at least one cl:;ss (year), and perhaps 
with data for random samples of all other classes 
(years). Results of the analysis should be 
evaluated and examined for agreement. If substantial 



agreement among analysts exists, the classification 
system may be regarded as reliably established. If 
only one individual is given the bid, he should be 
• required to make two or more analyses using different 
approaches . 

3. Experimental de^signs should be modified to incorporate 
types based on the analysis. Every effort should 
be made to represent the major types which cover the 
bulk of the mi'^shipman population. 
It should not be difficult to attract advanced investi- 
gators to the problem, since situations where the data is 
automatically provided, and where a typology is a realistic 
working hypothesis and a useful goal are rather rare. At 
least some of these individuals should be interested in the 
opportunity to apply, advance, and compare experimentally 
their methodological approaches to the problems of classifica 
tory analysis, while being supported financially in their 
effort. 

In the next se- ion is described a brief general survey 
of the problem of classificatory analysis, serving to point 
up critical areas of uncertainty and decision which must be 
dealt with in the proposed plans. A series of questions 
are suggested to be posed in the request for proposal. 
The worth of the results will depend critically on how these 
questions are handled by the investigators, and therefore 
will provide a basis for evaluation of the proposals. 



II. Problems in Typological Analysis 

Techniques of typological analysis, with few 'exceptions, 
•re based on the premise tha^ the resemblence of profiles 
or score patterns can be reduced to a single quantitative 
measure of similarity. Given a measure of .'Similarity of each 
pair of persons in the sample, methods of t/pological analysis 
are applied to identify groups of persons which have high 
intragroup similarity and at the same time, low similarity 
to members of other groups. There are myriad variations of 
this basic approach, many of which will be indicated and 
briefly discussed below. A seiies of questions" will then be 
suggested to be posed 'in the request for proposal. 
A. Alternative Methods 

Typological solutions of similarity measures may be based 
on analytic or structural Methods. The analytic methods 
include Q-technique factor analysis (Cattel, 1952), multl.- 
dimensional scaling (Shepard and Carrol, 1966), and latent 
profile analysis (Gibson, 1956) which is based upon latent 
structure analysis. The analytical methods employ explicit 
mathematical manipulations to achieve a solution. 

In the case of a Q-techn.*que factor analysis, persons are 
treated as tests, and using any of the t*.ommon methods of factor 
extraction and rotation, factors are derived representing 
ideal types of persons. Factor loadings for an individual 
repre'^ent similarities to each type. The hope is to explain 
the similarities among people in terms of their relations to 



a minimal number of ideal types, just as ordinary factor 
analysis explains the correlation among tests in terms of 
their factorial composition. The approach of the multi- 
dimensional scaling technique is similar, but only assumes 
that the similarity measures are monotonic iv relation to 
measures of true similarity. By permitting monotonic dis- 
tortion of the similarity scale, the number of dimensions 
required to explain the similarities can be minimized. Both 
factor analysis and multi-dimensional scaling recognize the 
existence of individuals which are intermediate between the 
ideal types, mixing features of two or more types. 

Latent profile analysis, on the other hand, attempts to 
explain the correlations among tests in terms of the number 
of individuals which come from seVeral subpopulations having 
different average (latent) profiles across tests. Inter- 
mediate individuals are not recognized, since the deviation 
from the group-average (ideal) profile is considered error. 

Structural methods are based on various kinds of rules 
for grouping or clustering similar individuals and separating 
dissimilar individuals. The techniques are similar to those 
of quantitative biological taxonomy, and tend to be rather 
arbitrary and "artful." These methods include the computer- 
based "taxonome" (Cattel, et. al., 1966), simple clustering 
methods (Fortier an4 Solomon, 1966), and a whole variety of 
hierarchical clustering, grouping or pattern analyses (Johnson, 
1967; McQuitty, 1957, 1960abc, .1961, 1963, 1964, 1966abcd, 
1967; Ward, 1963). The major distinctions among these methods 
have to dc with whether or not an overlapping hierarchy of 



types is conceived, and how clusters or groups are built 
up or defined. 

Despite their differences, it would be valuable in the 
present problem to be able to compare analytical and structural 
analyses of the same data. If some sensible relationship 
between the results could be established, it would provide _ 
considerable support for the existence of "real" types in 
the Academy population. 
B. Measures of Similarity 

Whether analytic or structural, the results of the analysis 
depend to a considerable extent on how the data is standardized 
and what kind of similarity measure is derived from the data. 

» 

The scores may be analyzed as raw scores, standardized in 
relation to an external population, standardized with respect 
to persons in the sample, standardized over tests, or even 
converted to ranks (Nunnally, 1962; Kendall, 1966). Which of 
these should be used is not at all clear, and some rationale 
must be developed for selection of one over the other. 

Given properly standardized data, the similarity measure 
may be based on the raw products, covariances, correlations 
(Nunnally, 1962), and may be corrected for chance association 
(Cattel, 1949). Invariance over variable reflection may be 
desired, in which case a measure of Euclidian distance CChronbach 
and Gleser, 1953) or other similarity measure (Cohen, 1969) 
may be used having this property. Some rationale is also 
required for the selection of an appropriate measure. 
C. Evaluation of Reliability 

To assess the reliability of the findings, analysis of two 
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or more subgroups of the Academy population must be conducted 
and compared. The number of such groups, their, size, and the 
method to be used in assessing the replicability of results 
is another area of uncertainty. 

A second problem of reliability has to do with the assign- 
ment of individuals to a category in the taxonomic system. 
Clearly, if the taxonomic categories are to be used as a basis 
of experimental stratification, one must have some method 
of selecting individuals to .represent the type, and also to 
have some assurance that they are in fact representative of 
that type. Further, if the typology is to '.^e used for other 
descriptive or administrative purposes, the problem of 
intermediate or unclassif iable individuals must also be explicitly 
recognized and dealt with on a rational basis. 
D. Request for Proposal 

The following questions, based upon considerations raised 
above, should be posed in a request for proposal. 

1. What methods of analysis or grouping rules are to be 
used in the derivation of the taxonomic system? T^at are 
the advantages and disadvantages of these methods in relation 
to other methods of achieving the same ends? 

2. How are the results to be interpreted in psychological 
and behavioral terms? 

3. How are. the test scores to be standardized and used to 
derive measures of similarity or distance? What are the 
advantages and disadvantages of such measures in relation 
to other alternative measures? 



-11- 



4. How is the reliability of the findings to be assessed? 
What is the number and size of subgroups to be used for 
cross-validation of results? How is the correspondence 

of results between subgroups to be evaluated? 

5. How is an individual to be classified into a particular 
category of the finished taxonomic system? What are the 
chances and effects of misclassif ication? How are 
intermediate or unclassif iable individuals to be handled 

in the system? 
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